perm filename VAXTIM[TIM,LSP]1 blob
sn#715165 filedate 1983-06-16 generic text, type C, neo UTF8
COMMENT ⊗ VALID 00010 PAGES
C REC PAGE DESCRIPTION
C00001 00001
C00002 00002 SCCPP Franz
C00007 00003 ∂06-Jul-82 1703 Kim.fateman at Berkeley Re: Progress
C00009 00004 ∂26-Feb-81 2227 CSVAX.fateman at Berkeley
C00023 00005 ∂06-Jul-82 1724 ARPAVAX.fateman at Berkeley
C00025 00006 780 Franz
C00031 00007 ∂08-May-82 2148 Kim.jkf at Berkeley updated mas benchmark results
C00032 00008 Franz
C00033 00009 Barrow's FFT times
C00035 00010 The INTERLISP VAX test
C00038 ENDMK
C⊗;
SCCPP Franz
∂11-Apr-81 1001 CSVAX.jkf at Berkeley result of pairs benchmark on franz.
Date: 11 Apr 1981 09:56:26-PST
From: CSVAX.jkf at Berkeley
To: rpg@su-ai
Subject: result of pairs benchmark on franz.
pair benchmark results
submitted by j foderaro
(csvax.jkf@berkeley)
10 april 81
Here are the results or running the pairs benchmark on Franz Lisp on
a VAX 11/780 runing Berkeley 4BSD Unix. The load average was less
than one when the timing was done. Timing on Unix is done in 60ths of
a second and the time charged to a process includes some of the system
overhead for memory management. I ran the benchmarks on an unloaded
system to reduce the interference of the memory manager.
The program was run with modification only to the test function
shown below. Perhaps you should in the future use macros like (cpu-time)
and (gc-time) and each site can define these macros.
(defun test ()
((lambda (t1 x gt)
(setq x (pairs a b () 'equal () () ()))
(setq t1 (- #-Franz (runtime)
#+Franz (car (ptime))
t1))
(setq gt (- #-Franz (status gctime)
#+Franz (cadr (ptime))
gt))
(print (length x))
(print (list 'runtime
(QUOTIENT (FLOAT (- t1 gt))
#-Franz 1000000.
#+Franz 60.)))
(print (list 'gctime
(quotient (float gt)
#-Franz 1000000.
#+Franz 60.)))
#+Franz (terpri))
#-Franz (runtime) #+Franz (car (ptime))
()
#-Franz (status gctime)
#+Franz (cadr (ptime))))
---
The size of the compiled file is 2768 bytes.
Here are the results::
Script started on Fri Apr 10 22:16:58 1981
Reval: lisp
Franz Lisp, Opus 34
-> (load 'pairs)
[fasl pairs.o]
t
-> (test)
2592(runtime 7.033333333333333)(gctime 23.53333333333333)
nil
-> (test)
2592(runtime 7.383333333333333)(gctime 4.816666666666667)
nil
-> (test)
2592(runtime 7.283333333333333)(gctime 4.366666666666667)
nil
-> (test)
2592(runtime 7.333333333333333)(gctime 4.666666666666667)
nil
-> (exit)
------
I looked at which functions were being called and it seems just about all this
benchmark does is call 'member' 10,000 times. I noticed that in our system
'memq' would do as good a job as 'member' so I replaced member by memq
and ran it with these results:
Reval: lisp
Franz Lisp, Opus 34
-> (load 'memqpairs)
[fasl memqpairs.o]
t
-> (test)
2592(runtime 1.683333333333333)(gctime 23.55)
nil
-> (test)
2592(runtime 1.733333333333333)(gctime 4.833333333333333)
nil
-> (test)
2592(runtime 1.766666666666667)(gctime 4.35)
nil
-> (test)
2592(runtime 1.783333333333333)(gctime 4.7)
nil
-> (exit)
script done on Fri Apr 10 22:21:50 1981
∂06-Jul-82 1703 Kim.fateman at Berkeley Re: Progress
Date: 6 Jul 1982 16:49:48-PDT
From: Kim.fateman at Berkeley
To: RPG@SU-AI
Subject: Re: Progress
Cc: Kim.jkf@Berkeley
I just ran DERIV; Franz Lisp opus 38.20 on a vax 11/780 running with
a load average of between .5 and .8 (light). Recorded time 27 seconds
of which 16 were in garbage collection.
The only change to the file was to declare (localf der1 deriv).
jkf: get back to your thesis.
∂06-Jul-82 1740 ARPAVAX.fateman at Berkeley
Date: 6 Jul 1982 17:35:21-PDT
From: ARPAVAX.fateman at Berkeley
To: rpg@su-ai
Cc: ARPAVAX.jkf@Berkeley
jkf reminded me to do (sstatus translink on) to get faster times,
so with that, on a 780:
deriv = 25,2, 16.9 of which is GC
dderiv = 27.4, 17.6 of which is GC
fdderiv requires subrcall which we don't have exactly. I haven't
looked to see how funcall can work instead.
∂06-Jul->2 1802 Kim.fateman at Berkeley deriv
Date: 6 Jul 1982 17:57:22-PDT
From: Kim.fateman at Berkeley
To: rpg@su-ai
Subject: deriv
Cc: Kim.jkf@Berkeley
for a (load average 4) vax 750, no floating point accelerator, Franz opus 38.20
36.06 of which 22.2 is in GC.
I think a lighter load on the 750 would help somewhat.
∂26-Feb-81 2227 CSVAX.fateman at Berkeley
Date: 26 Feb 1981 14:42:52-PST
From: CSVAX.fateman at Berkeley
To: CSVAX.jkf@Berkeley, jlk@mit-mc, lisp-forum@mit-mc, rz@mit-mc
Cc: CSVAX.fateman@Berkeley
←←←←←←←←←←←←←←←←←←←←←←←←←←←←←←←←←←←←←←←←←←←←←←←←←←←←←←←←←
| | UCILISP | INTERLISP | MACLISP |Franz/VAX|
|-------------+---------+-----------+---------+---------|
| Interpreter | 57.0 | 26.0 | 22.8 | 65.0 |
|-------------+---------+-----------+---------+---------|
| Compiler | 2.90 | 15.0 | 0.69 | 1.1 ** |
|-------------+---------+-----------+---------+---------|
Times are for (TAK 4 2 0), where TAK is an interesting function
defined by Mr. Ikuo Takeuchi.
(DEFUN TAK (X Y Z)
(COND ((GREATERP X Y)
(TAK (TAK (SUB1 X) Y Z)
(TAK (SUB1 Y) Z X)
(TAK (SUB1 Z) X Y) ))
(T Y) ))
(**) 5.3 with (1- x) etc [no other declarations, so greaterp is closed comp.]
4.1 with local function declaration (fast subroutine call)
1.1 with > open compiled
times on a VAX 11/780 at Berkeley, Feb. 26, 1981
∂21-Feb-82 2132 Kim.fateman at Berkeley (tak 18. 12. 6.)
Date: 21 Feb 1982 18:36:21-PST
From: Kim.fateman at Berkeley
To: rpg@su-ai
Subject: (tak 18. 12. 6.)
seems to take about 13.3 seconds using Franz opus 38, with tak
compiled as a local function, and 21.3 seconds compiled in the normal
fashion. (Any function can be compiled as a local function, but it
takes more room if a local function must also be callable from interpreted
code.)
Run on a vax 11/780 with a load average of about 2.5.
Tail merging was done by the compiler to eliminate the direct recursion.
∂21-Feb-82 2134 ARPAVAX.fateman at Berkeley oops
Date: 21 Feb 1982 19:49:54-PST
From: ARPAVAX.fateman at Berkeley
To: rpg@su-ai
Subject: oops
Something funny with those last numbers. I ran it again, this
time correctly timing it, (I hope) and I got 1.13 seconds for
local function, 8.4 seconds for normal. This still seems strange,
though.
I also timed interlisp VAX, and got about 4.5 seconds, compiled.
∂21-Feb-82 2134 CSVAX.fateman at Berkeley explanation for the different times....
Date: 21 Feb 1982 20:36:38-PST
From: CSVAX.fateman at Berkeley
To: rpg@su-ai
Subject: explanation for the different times....
(< x y) is not open coded unless the compiler can be sure both
arguments are fixnums. (< in maclisp works by accident on floats,
also). (<& x y) is the right thing to do, where the "&" is from
common lisp.
∂21-Feb-82 2139 Kim.fateman at Berkeley more on (tak 18 12 6)
Date: 21 Feb 1982 21:13:08-PST
From: Kim.fateman at Berkeley
To: rpg@su-ai
Subject: more on (tak 18 12 6)
Cc: Kim.jkf@Berkeley
Franz, interpreted, runs it in 77 seconds, Interlisp VAX interpreted
in 199.5 seconds. The interlisp compiler produces some mysterious
code, but it is clear that it also removes the tail-recursive call.
(It seems that interlisp open-compiles sub1 to a "ROTL". Since
the Franz compiler does not specialize 1- to fixnums (it must run on
bignums, too), it seems interlisp has the benefit of additional
optimization.
I look forward to your list of data points on this timing.
∂23-Feb-82 0234 Kim.fateman at Berkeley gack, another data-point on TAK
Date: 22 Feb 1982 19:49:49-PST
From: Kim.fateman at Berkeley
To: rpg@su-ai
Subject: gack, another data-point on TAK
John Foderaro pointed out to me that I was running the timing with
(status translink)= nil , which is the debugging setting. It doesn't
really affect the local function, but it makes a big difference for the
"normal compile" version. It reduces it from about 8 seconds to 2.13 seconds,
which perhaps also gives you an idea of function-linkage overhead if
all calls go through the transfer table.
Thus the "best" time is still about 1.13 seconds (localfunction),
then 2.13 (normal function, translink = t), then the other data
as given. (This is for Franz Lisp, on a vax 11/780.)
Well
I must commend you and GJC for providing good results. Both you,
he, and a third party provided results for Franz that are the same to the
second decimal place. I take this as honesty all around. In any event,
for the record, could you please explain exactly the implementational
difference between TRANSLINK=T, translink=(), and the localfunction compilation?
I know what they are in general terms, but a detailed explanation would
be good. Thanks.
-rpg-
∂23-Feb-82 1455 Kim.fateman at Berkeley translink etc
Date: 23 Feb 1982 14:54:51-PST
From: Kim.fateman at Berkeley
To: rpg@su-ai
Subject: translink etc
Cc: Kim.fateman@Berkeley
Local function compilation is the simplest to explain:
a function invocation can be implemented by a jsb directly to
an entry point, if enough information is known at compile time.
This totally inhibits debugging, generally hides the name of the
local function from functions not compiled "at the same time", and
is very fast. in Franz one does (declare (localf tak)) for example.
translink: The first time a function foo is called,
a reference through a table of transfer vector pairs <name, location> is made,
by jumping to "location" associated with foo. Initially this pair is
<foo, qlinker>
Qlinker is a general calling routine which refers to the atom's (foo's)
function definition cell, accessing its definition. If the definition
is binary code, say bcd←foo then the pair in the transfer table is updated,
if translink = t, to be <foo, bcd←foo>. The function is then invoked.
The next time foo is invoked, the branch to Qlinker is avoided, and it
goes much faster.
If foo is not compiled, or if translink is NIL, then the transfer table
remains unchanged, and the overhead of going through qlinker is present
each time foo is called.
Setting translink to t leaves somewhat less information on the stack, so
the backtrace function has to work harder to find out what happened. Also,
if foo's definition cell is changed, a re-linking has to be done.
There is an analogous kind of thing in interlisp, called linked
and non-linked functions.
∂26-Apr-82 0053 Kim.jkf at Berkeley franz tak benchmarks
Date: 26 Apr 1982 00:40:32-PDT
From: Kim.jkf at Berkeley
To: rpg@su-ai
Subject: franz tak benchmarks
Cc: Kim.fateman@Berkeley, Kim.jkf@Berkeley
There seems to be a misunderstanding about the possible function linkages
in Franz Lisp. Your report lists, 'normal function call',
and 'fast function call'. Franz lisp users would call these 'slow
function call' and 'normal function call'. There are modes which
are even slower than 'slow function call'. You can (*rset t) and
get 'real slow function call' or bind evalhook and get
'real real slow function call'. From experience I can say that the
normal mode for franz users with compiled code is to set the
linkages on (what your report called 'fast function call'). Since
I believe the purpose of the lisp timing project was to compare lisps
as they are typically run, I hope that you use our 'linkages on' numbers
for comparison with other systems.
Your note about 'local function call' is also wrong. A function
declared local is not examined by the compiler in any greater detail
than any other function. The only requirement is that a function declared
local actually appear in the same file and that it not be a lexpr.
I re-ran all timings you requested tonight. The lisp system and
compiler which produced these timings is available to anyone who
asks for it. The source is sitting on our c70 arpanet host.
I want to thank you for taking time to organize the collection of
timing information. I find the results very interesting.
Tak benchmarks run 26 april 82 by jkf
Lisp Opus 38.13, Liszt 8.05
(cpu time in secs, no gc's occurred)
Link type 780 750
--------- ------ ------
generic slow 28.6 51.0
arithmetic normal 7.8 11.6
fixnum slow 8.3 14.8
arithmetic normal 2.1 3.3
local function n/a 1.1 1.9
calls
------
takf n/a 4.3 6.7
(with funcalls)
Key: slow = (sstatus translink nil), fasl = (sstatus translink on)
n/a = doesn't make any difference how the links are set.
The generic example uses add1, sub1, lessp. All others use the
fixnum specific operations.
takf uses funcall rather than the normal way of function calling.
Franz
Your note was well taken. I have updated my notes accordingly. Could you
please explain in more detail what the local function call accomplishes?
Perhaps a quick description of the runtime stack and some code comparing the
results of normal function call and local function call would help me
understand (and explain it to others) what's up. Thanks.
Also I will include your new benchmark in the crop.
-rpg-
∂06-Jul-82 1724 ARPAVAX.fateman at Berkeley
Date: 6 Jul 1982 17:02:05-PDT
From: ARPAVAX.fateman at Berkeley
To: rpg@su-ai
Cc: ARPAVAX.jkf@Berkeley
dderiv under the same conditions, 30.8 sec, 17.28 in GC.
[same conditions =
I just ran DERIV; Franz Lisp opus 38.20 on a vax 11/780 running with
a load average of between .5 and .8 (light). Recorded time 27 seconds
of which 16 were in garbage collection.
The only change to the file was to declare (localf der1 deriv).]
∂06-Jul-82 1740 ARPAVAX.fateman at Berkeley
Date: 6 Jul 1982 17:35:21-PDT
From: ARPAVAX.fateman at Berkeley
To: rpg@su-ai
Cc: ARPAVAX.jkf@Berkeley
jkf reminded me to do (sstatus translink on) to get faster times,
so with that, on a 780:
deriv = 25,2, 16.9 of which is GC
dderiv = 27.4, 17.6 of which is GC
fdderiv requires subrcall which we don't have exactly. I haven't
looked to see how funcall can work instead.
;;; 780 Franz
Here are the results or running the pairs benchmark on Franz Lisp on
a VAX 11/780 runing Berkeley 4BSD Unix. The load average was less
than one when the timing was done. These results supersede the timings
you have which were made by us in March.
-> (setup)
(z 1 1.0 0 (y 1 1.0 0 (x 1 1.0 0 1.0)))
-> (bench 2)
(power= 2 (0.0167 0.0) (0.0333 0.0) (0.0 0.0))
-> (bench 5)
(power= 5 (0.15 0.0) (0.75 0.4333) (0.3833 0.2166))
-> (bench 10)
(power= 10 (2.8167 1.05) (8.2333 3.3) (3.2 1.2333))
-> (bench 15)
(power= 15 (18.2333 5.35) (92.0333 41.6333) (18.8 5.1333))
->
script done on Fri Apr 24 06:07:48 1981
;;; Franz on 11/780 and 11/750
∂25-May-82 0907 jkf@ucbkim at Berkeley frpoly benchmark, complete results
Date: 25 May 1982 09:03:10-PDT
From: jkf@ucbkim at Berkeley
Mail-From: ucbkim received by UCBVAX at 25-May-82 08:46:04-PDT (Tue)
Date: 25-May-82 08:52:21-PDT (Tue)
From: jkf@ucbkim
Subject: frpoly benchmark, complete results
Via: ucbkim.EtherNet (V3.100 [3/27/82]); 25-May-82 08:52:27-PDT (Tue)
Via: ucbvax.EtherNet (V3.100 [3/27/82]); 25-May-82 08:46:04-PDT (Tue)
To: rpg@su-ai
Cc: jkf@fateman@Berkeley
frpoly benchmark -[Tue May 25 08:51:42 1982 by jkf]-
Results for Franz Lisp Opus 38.18, and compiler Liszt 8.07
11/780
Slow links:
(power= 2 (0.05 0.0) (0.06666666666666667 0.0) (0.06666666666666667 0.0))
(power= 5 (0.6 0.0) (1.18333 0.4166666666666667) (0.9833333333333333 0.4))
(power= 10 (7.6666667 0.8833) (14.866667 3.83333) (8.6166667 1.433333333333))
(power= 15 (52.51666666666667 5.316666667) (132.8 43.0) (54.2666667 6.95))
Normal links:
(power= 2 (0.01666667 0.0) (0.01666666667 0.0) (0.03333333333333333 0.0))
(power= 5 (0.2 0.0) (0.3833333333333333 0.0) (0.2166666666666667 0.0))
(power= 10 (2.333333333333333 0.0) (8.0 2.266666666666667) (2.6 0.0))
(power= 15 (20.6 4.016666666667) (86.5 30.3833333) (25.0833333 6.766666667))
Local Functions:
(power= 2 (0.01666666667 0.0) (0.016666667 0.0) (0.01666666666666667 0.0))
(power= 5 (0.15 0.0) (0.66666667 0.36666667) (0.51666667 0.3333333333333333))
(power= 10 (2.6 0.8333333333333333) (8.866667 3.7) (3.85 1.833333333333333))
(power= 15 (18.48333333 5.6333333) (93.866667 41.75) (21.7 6.966666666667))
11/750
Slow links:
(power= 2 (0.0833333333333 0.0) (0.116666666667 0.0) (0.08333333333333 0.0))
(power= 5 (0.933333333 0.0) (1.8 0.56666666667) (1.6166666667 0.633333333333))
(power= 10 (12.06666667 1.3666666667) (22.26667 5.93333) (13.86667 2.8833333))
(power= 15 (82.5666667 9.3833333) (202.683333 67.46666667) (86.83333 11.2167))
Normal links:
(power= 2 (0.05 0.0) (0.03333333333333333 0.0) (0.03333333333333333 0.0))
(power= 5 (0.35 0.0) (1.18333333 0.566667) (1.0166667 0.6666666666666667))
(power= 10 (5.3666667 1.383333) (14.95 5.816667) (7.266666666666667 2.95))
(power= 15 (37.233333 9.1166667) (155.45 67.15) (41.46666666666667 11.0))
Local functions:
(power= 2 (0.0333333 0.0) (0.03333333333333333 0.0) (0.01666666666666667 0.0))
(power= 5 (0.2666666666666667 0.0) (0.4833333333333333 0.0) (0.3 0.0))
(power= 10 (3.15 0.0) (11.81666666666667 3.516666666666667) (3.466666666666667 0.0))
(power= 15 (29.46666666666667 6.65) (129.5666666666667 48.21666666666667) (36.7 11.21666666666667))nil
Slow links means (sstatus translink nil)
Fast links means (sstatus translink t)
Local functions means that certain functions are declared to be local
to this file.
∂08-May-82 2148 Kim.jkf at Berkeley updated mas benchmark results
Date: 8 May 1982 21:41:30-PDT
From: Kim.jkf at Berkeley
To: rpg@su-ai
Subject: updated mas benchmark results
Cc: Kim.fateman@Berkeley
I ran it with local functions and here are the complete results:
mas benchmark (tak benchmark using tak0 through tak99)
Lisp Opus 38.16,
Liszt 8.06
780 750
slow links 13.26 19.17
normal links 3.62 5.09
local functions 1.7 2.7
slow links = translink nil, fast links = translink on
;;; Franz
I think they mean TAKR here.
∂08-May-82 2148 Kim.jkf at Berkeley updated mas benchmark results
Date: 8 May 1982 21:41:30-PDT
From: Kim.jkf at Berkeley
To: rpg@su-ai
Subject: updated mas benchmark results
Cc: Kim.fateman@Berkeley
I ran it with local functions and here are the complete results:
mas benchmark (tak benchmark using tak0 through tak99)
Lisp Opus 38.16,
Liszt 8.06
780 750
slow links 13.26 19.17
normal links 3.62 5.09
local functions 1.7 2.7
slow links = translink nil, fast links = translink on
;;; Barrow's FFT times
---------------------------------------------------------------
| Machine & | Interpreted | Compiled | Interpreted |
| Language | Secs Ratio | Secs Ratio | /Compiled |
---------------------------------------------------------------
| | | | |
| 2060 | 28.7 1.0 | 0.532 1.0 | 53.9 |
| Maclisp | | | |
| | | | |
| LispM | 97.2 3.39 | 3.52 6.62 | 27.6 |
| Zetalisp | | | |
| | | | |
| Vax | 135.5 4.72 | 66.5 125.0 | 2.04 |
| Franzlisp | | | |
| | | | |
| 2060 | 37.3 1.30 | 12.6 23.68 | 2.96 |
| Interlisp | | | |
| | | | |
| Dolphin | 431.7 15.0 | 149.1 280.3 | 1.54 |
| Interlisp | | | |
| | | | |
---------------------------------------------------------------
-------
The INTERLISP VAX test
Mail-from: ARPANET site USC-ISIB rcvd at 23-Jul-82 1108-CDT
Date: 23 Jul 1982 0907-PDT
Sender: RBATES at USC-ISIB
Subject: Re: lisp comparison.
From: Raymond Bates <RBATES at ISIB>
To: CL.BOYER at UTEXAS-20
Cc: CMP.GOOD at UTEXAS-20, ddyer at USC-ISIB, lynch at USC-ISIB,
Cc: Novak at SUMEX-AIM
Message-ID: <[USC-ISIB]23-Jul-82 09:07:07.RBATES>
In-Reply-To: Your message of Thursday, 22 July 1982 13:12-CDT
*** EOOH ***
Date: Friday, 23 July 1982 11:07-CDT
From: Raymond Bates <RBATES at ISIB>
Sender: RBATES at USC-ISIB
To: CL.BOYER
cc: CMP.GOOD, ddyer at USC-ISIB, lynch at USC-ISIB, Novak at SUMEX-AIM
Re: lisp comparison.
Here is the DRIBBLE file of a long run:
NIL
4←(GCGAG NIL]
40
5←LOAD(IREWRITE.v]
compiled on 22-JUL-82 09:22:17
File Created: 5-Jul-82 12:52:49
% warning! this file possibly incompatible!
Compiled by version 3.0 but current version is 2.1
IREWRITECOMS
/lisp/rbates/lisp/IREWRITE.v;2
6←(SETUP]
T
7←(FOR I FROM 1 TO 20 DO (PRINT (TEST ]
(T 76048 0)
(T 76464 0)
(T 76496 0)
(T 76272 0)
(T 89072 4784)
(T 80992 0)
(T 81840 0)
(T 81600 0)
(T 90512 4752)
(T 81904 0)
(T 82032 0)
(T 81248 0)
(T 90512 5456)
(T 80768 0)
(T 80800 0)
(T 80656 0)
(T 91424 5456)
(T 82112 0)
(T 82384 0)
(T 82432 0)
NIL
8←(DRIBBLE]
You many wonder why the cpu time has an extra 10 seconds each
time the system does a gc. The reason is that the system has to
re-hash all the hash array after the gc and time is being charge
to the computation time and not gc time (we will fix that). This
test was done on our 780. Any thing else you would like to see
or known?
/Ray
∨